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ABSTRACT 

Because the Navy may be forced to use larger numbers 
of marginalrv qualified recruits in the future and thus may have to 
expand its academic remedial training (ART) program, a study was 
conducted to compare the instructional and cost effectiveness of the 
standard classroom approach to ART /ith the use of a computer 
assisted instructional (CAI) system known as the Performance-related 
Enabling Skills Training (PREST) program. PREST, which focuses on 
improving functional reading and study skills, was used with a 
Control Data Corporation (CDC) PLATO computer system. A total of 152 
recruits at Orlando, Florida, participated in the evaluation, with 75 
receiving ART through classroom instruction and 77 through PREST. It 
was found that: (1) the two groups were equal in instructional 
benefits in terms of immediate and prolonged improvement; (2) the 
PREST configuration was less cost effective than the classroom 
approach in 1980; and (3) if civilian instructors were used and 
computer usage charges declined by 5% per year, PREST would become 
cost effective in Orlando in the late 1980's. It is recommended that 
the PREST program not be /substituted for the current ART classroom 
approach, but that consideration be given to replacing the vocabulary 
and comprehension ART modules with a computer-based system using 
low-cost stand-alone microcomputers as the delivery devices. These 
and other recommendations, study methodology and results, a.nd a 
20-item bibliography are included. (ESR) 
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FOREWORD 



This project was conducted in response to a Chief of Naval Education and Jraimng 
(N-5) request to evaluate the performance-related enabling skUls training (PR^IJ 
Droi'K:t, a computer-based approach to the Navy's academic remedial training (ART) 
proeram. The evaluation was conducted at the Recruit Training Command^ (RTC), 
Orlando, FL during FY80. Results are intended for use by the Chief of Naval Education 
and Training (N-5) and the RTCs. 

We gratefully acknowledge the assistance of CDR 3ames Kiefer, YN2 Pam Keskeys, 
and the other members of the Academic Remedial Training Division, RTC Orlando; 
Dr Ron Bucknam of th'^ National Institute of Education; Nancy Ashcroft of the Orange 
County (Florida) Board of Public Instruction; Joe Small and Dick Wall of Control Data 
Corporation; and 3ocA oe Stolte, Shirley Smith, and Keith Kershner of Research for Better 
Schools, Inc. 



JAMES F. KELLY, JR. ^^^^ ]'^^^^^ 

Commanding Officer Technical Director 
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SUMMARY 



Problem and Background 



KJhe Navy may be forced to use larger numbers of marginally qualified recruits in the 
future, because of manpower shortages. If this occurs, academic remedial training (ART) 
will have to be expanded, resulting in a greater number of instructors^ Automated 
instruction is an attractive alternative, since its use can reduce the number of instructors 
needed. Therefore, to determine the economic and technologicati feasibility of automated 
instruction for the ART program, the Chief of Naval Education and Training (N-5) 
contracted for the development amd test of a computer-based approach^ hereafter 
referred to as thd^erformance-related Enabling Skills Training (PREST) Program^ 



Objective 



/The objectives of this effort were to compare the instructional effectiveness of the\ '[ ' ^ 
PREST configuration with that of the standard classroom approach, quantify the dif- | 
f erences in effectiveness in terms of cost savings, chart the expected future costs of each I 
altei native as a function of time, and predict the point at which the alteimatives break / 



even. 
Approach 



A total of 152 recruits who were enrolled in the ART program at the Recniit Training 
Commamd at Orlando participated in the evaluation. Of these, 75, the control group, 
received ART through classroom instruction; and 77, the experimental group, through 
PREST. Because of a change in policy that applied during 2 monthv.when only PREST 
recruits were being tested, the overall entering reading grade level (RGL) average of 
these recruits was one-half grade higher than that of the classroom group. To compensate 
for this difference, analyses were conducted on test scores obtained by restricted 
samples, which included only those recruits with RGLs between 5.0 and 6.0. Also, for 
both groups, restricted and overall samples were compared on background measures, as 
well as on measures of immediate (during ART) and prolonged (during the 7-week recruit 
training period) effects of instruction types on Navy-related and general reading skills. 

Findings ♦ ' 

The two groups were equal in instructional benefits on immediate impact and on 
prolonged benefits, rfhe cost analysis revealed that the PREST configuration was less 
cost-effective than the classroom approach in 1980, but that the difference would steadily 
decline in future yearsJ The predicted point at which the alternatives break even depends 
on whether the instructors are military or civilian, and on whether the computer usage 
charge is predicted to remain constant or decline five percent per year, (if civilian 
instructors are used and the computer usage charge declines by five percent per year, 
PREST will become cost-effective in Orlando in the late 1980s.") 

Conclusions 

Although PREST is not now as cost-effective as the classroom approach to ART, it 
may become cost-effective in the future. Its cost-effectiveness can be influenced by 
advances in computer technology (e-g., the continued development of stand-alone sys- 
tems), the type of instructor used (i.e., military vs. civilian), and the relative costs (e.g., 
salaries, overhead costs) incurred for these instructors. 
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R ecom mendations 

1. The PREST program should not 
approach at this time. 

2. Consideration should be given to 
modules of ART with a computer-based s>. 
microcomputers as the delivery devices. 

3. The worksheets currently used with PRES\ ..lould be incorporated into the ART 
study skills module. 



t^. The Navy Recruit Reading Test should br developed further as * sd«twn device^ 



:uted for the current ART cl^rocm 

\ the vocabulary 2uid comprehension 
id (b) using low-cost, stand-alone 



for ART. 



-t .*.'^ *• 
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INTRODUCTION 



Problem 

The military prime enlistment pool (i.e., male high school graduates, aged 18 to 21 
years, in the higher mental categories) will decrease significantly during the next 20 years 
(Congressional Budset Gifice, 1977). Consequently, the military services may be forced 
to accept more r<rcruits from the lower mental categories. Past experience has 
demonstrate^^ cnat these personnel have lower reading abilities, are less useful in the 
service, -tind attrite more frequently than do personnel in the higher mental categories. 

The Navy currently accepts small numbers of marginally qualified recruits and uses 
uniformed instructors to teach them basic skills, primarily reading, as part of the 
academic remedial training (ART) program at the three Navy Recruit Training Commands 
(RTCs) (Orlando, FL, Great Lakes, IL, San Diego, CA). The purpose of ART is to improve 
a recruit's reading and study skills so that the recruit can successfully complete the 
academic portion of recruit training. There is a strong relationship between reading 
ability and success in military training. For example, Hoiberg, Hysham, and Berry (197'>), 
found that the exit reading grade level (RGL) of ART graduates was the best predictor of 
successful completion of their first year of Navy service. 

If manpower shortages force a lowering of enlistment standards, remedial programs 
like ART can be expanded to accommodate larger numbers of poorly qualified recruits. 
Doing so, however, will divert more instructors from their regular assignments. If the 
Navy must accept increasing numbers of less qualified enlistees and continue enrolling 
them in the ART program, alternatives to the use of military instructors, such as 
civilianization or automation, must be considered. At present, civilian ART instructors 
are used, to a Umited extent, at no cost to the Navy through local adult education 
programs. These arrangements, however, are subject to change owing to policy changes 
at the local level. Thus, automated instruction appears to be an attractive alternative to 
the standard classroom approach, with either civilian or military instructors. 

Background 

ART Program 

The current ART program was standardized and implemented in October 1978 and 
appears to be successful, with over 85 percent of its graduates successfully completing 
recruit training. The 1979 throughput was 3266 students, with over 10 percent being 
nonnative English speakers. 

The ART program is based on a diagnostic-prescriptive approach to remedial reading 
and is offered to recruits who score between the '^.O and 6.0 RGL on the Gates-MacGinitie 
Reading Test (1978), Level D, which is administered to all Navy recruits during their first 
week in training. Those scoring above 6.0 continue with recruit training and those scoring 
below 6.0 are given an alternate version of the test to confirm their initial low score. If 
recruits score below i^.O on the second test, they are referred to a Review Board. If they 
score between i^.O and 6.0, they are enrolled in ART. 

The ART entrants are given the Stanford Diagnostic Reading Test (SDRT), Brown 
Level (Karlsen, Madden, & Gardner, 1976), which diagnoses weaknesses in vocabulary, 
literal comprehension, inferential comprehension, phonetic analysis, and structural analy- 
sis. Instructional modules in each of these reading skUl areas, which were developed by 
the Memphis State University under contract from the Chief of Naval Education and 
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Trainine (CNET), are then assigned to ART entrants according to their diagnosed needs. 
A separate cla^ is available for each reading skiU area, and recruits must pass a 
criterion-referenced module test before they proceed to the next class. A typical class 
size is about 12 students per instructor. Recruits require from 1 to 5 weeks to complete 
the reading skUi modules, depending on the number assigned. 

After ART recruits complete their assigned reading skills modules, they must 
complete a study skills module, which prepares them for the academic curriculumof 
recruit training. (Continued failure in these academic tests can lead to discharge.) This 
study skills module is also required of recruits who were not selected for ART initially, 
but who failed one or niore academic tests given as part of recruit training. 

The confirmatory Gates-MacGinitie and diagnostic tests are administered by an ART 
staff member and the criterion tests, by the instructors. All tests, from the initial Gates- 
MacGinitie to the criterion-referenced module tests, are graded with an optical scanner 
linked to the Navy's Integrated Training and Resource Administration System in 
Millington, TN. Test administration and grading require one full-time staff member. 

Performance-related Enabling Skills TrainiPR (PREST) Program 

With costs of computer hardware decreasing at a steady pace, prospects for less 
expensive, computer-based instruction are improving (Wolfe & Williams, 1979). To 
examine these prospects, CNET contracted for tl>e development of a computer-based ART 
program thaffocused on improving Navy functional reading and study skills. The program 
developed entitled "performance-related enabiir^ skills training" (PREST), called lor on- 
line reading and study skills instruction via the Control Data Corporation (CDC) PLATO 
system, augmented by Navy-related off-line drill and practice. 

PLATO is a comprehensive, computer-based educational system that uses a central- 
ized-mainframe CDC CYBER 70, 170, or 6000 series computer. From this computer, 
instruction is downloaded over leased phone lines to the PLATO terminal, a micropro- 
cessor-based device with 16K of random-access memory avaUable for storage. A read- 
only memory of 2K bytes contains the operating system and diagnostic routines for 
hardware rnaintenance. The terminal provides high-resolution computer graphics, an 
electronic keyboard, and a touch-sensitive panel. 

The on-line portion of PREST consists of two introductory modules, 9^* reading 
modules (selected from among the 111 reading modules within the Basic Skills Learning 
System an adult basic skills curriculum from CDC's existing courseware library); and six 
speciall y developed study skills modules appUcable to academic needs during Navy recrtat 
training. The computer-based management system of PREST routes each recruit through 
on-Une instruction for all of the introductory and study skUls modules. For cn-line 
reading instruction, it preso: ibes and automatically assigns modules based on individual 
on-line diagr>osis. 

Off-line study skill application activities allow recruits to make a transition from 
learning via a computer to learning via reading assignments and classroom lectures, which 
is the learning mode used in recruit academic training. Supplemental off-Une reading driU 
and practice materials, including those provided with the Basic Skills Learning system and 
those speciaUy developed for PREST, are available to recruits who wish additional 
practice or who are waiting for terminal access to continue on-line, reading activities. 
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PREST was designed for use by one military instructor working with 2^ recruits 
having access to 12 PLATO terminals. The instructor's role is to answer recruit questions 
about the purpose of ART and the operation of the terminals clarify misunderstandings 
about what occurs on-line, monitor student progress, provide on-the-sfK5t instruction as 
needed, direct selected study skills application activities, and provide feedback on some 
off-line materials. 

The instructional objective of both PREST and the standard classroom approach to 
ART is to provide recruits with basic reading and study skills needed to complete 
successfully the academic portion of recruit trsdning. Thus, it is important to determine 
whether PREST is effective, in terms of both instruction and cost, as compared to the 
classroom aipproach to ART. 

Purpose 

The purposes of the evaluation reported here were to compare the instructional 
effectiveness of PREST with that of the standard classroom approach to ART, quantify 
differences between the alternatives in effectiveness in terms of cost savings, chart the 
expected future costs of each alternative as a function of time, and predict the point at 
which the alternatives break even. 



APPROACH 

Instructional Effectiveness 

Previous CAI cost-effectiveness evaluations have been criticized because they did 
not objectively monitor students in their follow-on duty assignments (Orlansky & String, 
1979). The studies reviev;ed by Orlansky and String relied on pre-post measures of student 
performance during training, instructional time, academic attrition, arid student and 
instructor attitudes. Although such measures are critical to assessing a program's 
instructional effectiveness, they do not address the more important operational issue of 
job preparedness. Also, postgraduate effectiveness measures often consist only of a 
supervisor's opinion, which can be influenced by factors unrelated to the training being 
evaluated. 

Because of these criticisms, the current evaluation was designed to compare both the 
immediate (during ART) and prolonged (during the 7-week recruit training period) effects 
of classroom and PREST instruction on Navy-related and general reading skills. 

Sample 

A total .of 152 recruits who were enrolled in the ART program at RTC Orlando 
participated in the evaluation. Of these, 75, the control group, received ART through 
classroom instruction; and 77, the experimental group, through PREST. All subjects were 
native English speakers. 

Because the number of recruits entering ART Orlando in late 1979 was less than 
expected, 55 control recruits were tested between October 1979 and January 1980, and 
the other 20, in 3une and July, 1980. The entire PREST group was tested between 
February and May, 1980. In iMarch and April 1980, when only PREST recruits were being 
tested, a temporary change in Navy policy increased the RGL cutoff score for ART 
selection from 6.0 to 6.7. This change permitted 21 recruits with RGLs in that range to 
enter the PREST group, which, meant, of course, that the overall entering RGL average 



of PREST recruits was higher than that of the classroom group. Further, the classroom 
group included more recruits reading below the 5.0 RGL than did the PREST group. 
Therefore, statistical- analyses were conducted on a restricted sample, which included only 
those recruits with RGLs between 5.0 and 6.0 (*7 recruits for both the classroom and 
PREST groups). 



Measures 



The following background data were coUected for each sample membet-: (1) the 
scores obtained on the Armed Forces Qualifying Test (AQFT) and <^ ^^'^.^''^^l 
(WK) subtest of the Armed Services Vocational Aptitude Battery (ASVAB), (2) entering 
RGL as measured by the confirmatory Gates-MacGinitie test (197S), and (3) high school 
graduation status. 

To measure improvements in Navy-related reading skills during ART, NAVPERS- 
RANDCEN developed two equivalent 30-itein forms of a Navy Recruit Reading Test 
(NRRT). The test items, which were mainly multiple-choice with an occasional s«itence 
completion item, were patterned after those In the four recruit academic tests that are 
administered during recruit training. For each version of the test, 10 questions were 
desiened to test recruits' ability to obtain information from the table of contents and 
index of the Blue Jackets Manual; and the other 20, to comprehend material in tiiree 
passages on military subjects.'^ Eioth forms had been pilot-tested at RTC, San Diego. 
Based on the resul^ of this test, a time limit of 23 minutes was judged as appropriate. 

The comprehension section of the two 1965 versions of the Gates-MacGinitie Reading 
Test was also used to measure improvement in general reading comprehension skills 
durine ART. Since ART students complete one of the two 197& versions of this test 
during the first week of training and the other to confirm their initial low score, usmg 
them for a posttest would have required administering one form a second time. 

The final measure of immediate improvement was the number of instructional days 
recruits in the control and experimental groups spent in ART. Allowances were made for 
times when recruits were under medical care or otherwise absent from the classroom. U 
applicable, reasons for discharge were recorded. 

The four recruit academic tests administered during recruit training were used to 
measure prolonged effects of the two forms of ART. Since the expressed purpose of ART 
is to enhance performance on the academic portion of recruit training, these tests were 
the best avaUable measure of ART'S effectiveness. The first three tests each contained 
50 items; and the final test, 150 items. 

Design and Procedure 

The design conformed to Campbell and Stanley's "Nonequivalent Control Group 
Design" (1966); that is, recruits were assigned to one group or the other, depending on the 
month when they entered the Navy. All subjects entering the ART program were 
administered one version of the NRRT and Form 1 .from the 1965 Gates-MacGinitie test. 
When they completed the program, they were administered the alternate version of the 




^Six passage;* from the Blue Jackets Manual or The Basic Mihtary Requirements 
manual were modified by shortening sentences and substituting easy wordsfor difficult 
words. Three of tfiese passages were included in one test version; and the other three, m 
the other. 
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NRRT and Form 2 of the Gates-MacGinitie test. The alternate NRRT forms were 
counterbalanced between pre-and posttests within each group. During recruit training, 
the raw scores they obtained on the four academic tests were recorded. If applicable, 
reasons for discharge were recorded. 

Analyses 

In both groups, restricted (5.0 to 6.0 RGL) and overall samples were compared on 
background measures, as well as on measures of immediate and prolonged effects. Also, 
for the restricted samples, analyses of variance (ANOVAs) were used to compare pre-and 
posttreatment NRRT and Gates-MacGinitie test scores, as weU as the number of 
instructional days required. The scores obtained by the restricted samples on the four 
recruit academic tests were compared using a 2 x ^ repeated measures ANOVA. 

Cost Effectiveness 

The costs of the classroom and PREST approaches to ART, computed in constant 
1980 dollars, were compared. For the classroom approach, it was determined that four 
instructors would be needed, based on a 1979 throughput in Orlando of 929 recruits* This 
computation assumed an average instructor-to-student ratio of 12, 12.5 days of ART 
instruction for the average recruit, 250 days per year avaUable for instruction, and a 
fixed, month-by-month flow of students through ART. (In reaUty, the throughput is far 
greater during the simmer months. This problem is addressed in the Discussion section 
below.) One adcfitional staff member would be required to administer and grade the 
diagnostic test required by the classroom approach, for a total of five military 
instructors. The cost for these instructors was computed using the billet costs for the 
typical ART instructor, an Yeoman ($W,588) (see Koehler, 1980). The costs for ART 
classroom materials (e.g., the Basic Specific Skills Series , dictionaries, Skim-and-Scan 
booklets, vocabulary books, etc) were obtained from supply records. These materials 
usually can be recycled for use by other recruits; the length of thdr usable life varies 
from 1 to about ^ years, for a cost of $218S, 

For PREST, costs were based on the utilization of 2k terminals and 2 instructors, 
since it was determined that these resources could handle a load equivalent to that 
projected for the classroom approach. The PftE$T-related expenses include those for the 
terminals, computer usage charges, hardware maintenance, and instructors. The costs for 
the terminals were based on a purchase plan, which, in the long run, is far more favorable 
than a lease option, and were aUocated over a 5-year economic life, which is considered a 
reasonable estimate of the physical/teclvioiogical life of a terminal. The cost of 
computer hardware has demonstrated a steady decline, which is predicted to continue 
(Noyce, 1977). According to the cost figures supplied by CDC, the cost of PLATO 
terminals has been decreasing about 10 percent per year. This decline was considered in 
the analysis. 

The costs for computer usage, which are tied to system operator and maintenance 
costs and profit motives are more difficult to predict. Two assumptions can be considered 
here: either the cost will remain constant, or it will decline at 5 percent per year. 

The expenditures that are common to both the classroom and PREST approaches 
(e.g., for classroom space, desks, overhead lights, etc.) were not included in the cost 
analysis. Further, costs for ART personnel, such as the division officer and a yeoman, 
were not included since these personnel would be needed regardless of the nstructioncil 
delivery used. A workbook by the Training Analysis and Evaluation Group, entitled 
Improving Your Navy Reading Skills , is used in each approach. Also, the costs of the 
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consumable test-answer sheets used for the diagnostic and criterion tests in the classroom 
approach (estimated at $300 per year) are approximately equal to the costs for 
reproducing sufficient numbers of the consummable, RBS-developed worksheets for 
PREST. The costs of developing the ART classroom curriculum and the specially- 
developed PREST materials were treated as "sunk costs." 

PREST does avoid" the cost of the half day required to administer the diagnostic test 
and assign an instructional program. PREST does this task on-line at the beginning of the 
Basic Skills Learning System curriculum. This amounts to a cost avoidance of $7022, 
which was computed by multiplying the yearly throughput by a recrtiit*s half-day salary. 

The cost-effectiveness analysis was based on a break-even analysis, an analytical 
technique used to study the relationship between alternate cost patterns (Fabrycky & 
Thuesen, 197ft). This analysis focuses on finding th^j value of a variable, time in this case, 
at which the economic desirability of two alternatives is equaL Costs for the classroom 
and PREST versions of ART were projected over a 10-year planning horizon. 

Since consideration is being given to civilianizing the program, cost estimates for 
civilian instructors were obtained from the Orange Ccninty (Florida) Board of Public 
Instruction, Adult Program, which currently sup^es ART Orlando with 3.5 civilian 
classroom instructors at no cost to the Navy. The annual cost for civilian instructors, 
including overhead, is $21,175. Using this figure to replace the military costs and the 
previously mentioned assumptions about usage charges^ costs for the two approaches were 
projected thorugh 1989. 



RESULTS 

Instructional Effectiveness 

The mean measures obtained for the classroom and PREST groups are presented in 
Table 1. As shown, the mean entering RGL, which was based on lAte confirmaiory Gates- 
MacGinitie test, is one-half grade higher for the total PREST group than for the total 
classroom group. As indicated previouisly, this difference is attributad^le to the fact that 
21 PREST students entered the ART program with an entering RGL between 6.0 and 6.7. 
Within the restricted sample, the entering RGLs for the cleosroom and^i^REST groups 
were comparable. Althou^ the PREST restricted group had highw AFQT and WK scores 
than did the classroom group, the difference(\^ere not significant. 

An ANOVA performed on scores obtained by the restricted samples on the NRRT pre- 
and posttests showed that both groups improved si^ificantiy in their general reading 
comprehensive skills during the instructional treatment (F(l,7*) = 82.0, p < .001). 
However, there was no overall significant difference between the groups (F(l,7*) » 2.96, p 
> .05) and, more importantly, no significant difference in pre- to post^ain between groups 
(F(l,7*) = 0.5). These findings were verified by results of a duplicate ANOVA performed 
on the Gates-MacGinitie test scores obtained by the two groups. Confirmation came from 
an analysis of covariance (ANCOVA) performed on the post-NRRT scores using the pre- 
NRRT scores as a covariate. With pretest differences removed, there was no statistically 




Significant difference between groups on the post-NRRT scores (F(l,72) = 1.18, p > .25).^ 
Alio, the difference in instructional days required by the restricted groups was not 
significant (t - -82, p >^)- 

Table 1 

Measures Obtained for Restricted 
and Overall Samples 



Restricted Sample Overall Sample 

Classroom PREST Classroom PREST 
(N=^7) (N=^7) (N=75) (N=77) 



Background Data 





5.5 


5.6 


5.2 


5, 


A FOT ^cnr^ 


38.2 




38.8 


40, 


WK score 


if3.9 


ifif.3 




44, 


Hifcti School Graduates (%) 


96.0 


96.0 


93.0 


92, 


Immediate Effects 










NRRT score 










Pretest 


17.0 


17.6 


16.0 


I — 


Posttest 


20.1 


21.1 


19.3 




b 

Gates-MacGinitie test score 










Pretest 


39.7 


'♦O.l 






Posttest 


42.2 


43.3 






Number instructional days 


11.3 


12.2 


13 J', 


i2, 


Prolonged Effects 










Recruit Academic Test 1 (% correct) 


66 


66 - 


bJ 


68 


Recruit Academic Test 2 (% correct) 


63 


67 


63 


66 


Recruit Academic Test 3 (% correct) 


63 




63 


66 


F^ecruit Academic Test k (% correct) 


71 


71 


71 


71 



.6 



^Includes only those students with RGLs between 5.0 and 6.0. 

^Scores not included for the overall samples because of the ceiling effect noted. 



Measure 
(Mean) 



^The NRRT data were analyzed dy two methods, both yielding the same conclusion. 
They were: (i) a 2 x 2 x 2 repeated measures ANOVA with groups and the blocking-factor 
test forms as between effects an-:! vhe tests as repeated measures, and (2) a groups by test 
forms ANCOVA on the posttest scores using the estimated true pretest score as the 
covariate. Each of these designs has its advantages for the analysis qf intact nonrandom 
groups (Lord, 1967; Kenney, 1975; Cronback & Furby, 1970). The estimated true pretest 
scores were used to reduce bias in the ANCOVA caused by error in obtained pretest scores 
(Reichardt, 1979; Lord, 1956). The estimated true scores were calculated by multiplying 
deviation scores by the test-form reliability (Kuder-Richardson 20), and then converting 
back to raw scores. 
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. The ANOVA performed on recruit academic test scores obtained by restricted sample 
members completing all four tests revealed no overall significant differences between 
groups (F(l,6^^) = 1.79, p > .15) and no significant differential change oetween groups over 
tests (F(3,192) = 1.74, y > .15). Both groups improved significantly from Test 11 to Test 4 
(F(3.192) = 13.99, p < .001), raising their mean score from 66 to 71. Thus, th^ prolonged 
benefits provided by the two approaches to ART were equal, with the two groups havmg 
equal scores and gaining by the same amount* 

Finally, the attrition of the groups did not differ significantly, with 17 and 16 from 
the classroom and PREST groups respectively attriting before completion of recruit 
training. Most of the attrition was attributable to nonacademic reasons; only three 
recruits (one from the classroom group and two from the PREST group) werie discharged 
because of academic failure in recruit trainirig. 

Since the two approaches were equally effective, it was not necessary, to quantify any 
differences in instructional acjvantage in terms of cost savings* However, there are other 
aspects of PREST worth reviewing. Observations of the students using the PLATO system 
revealed that they were very attentive and very interested in their training and that one 
instructor could adequately handle 2k students* (The two military personnel at Orlando 
who served as the PREST instructors stated that 30 students would be a realistic 
maximum.) Most of the instructor time was spent in answering questions, providing 
feedback, or moving a student from on-line instruction to off-line materials. Al-fliough 
the efficacy of the off-line materials cannot be statistically assessed from a^^ailable data, 
the work-sheets developed by R^S appeared to be quite good and Navy relevant. 

Cost Effectiveness 



The costs for one year, 1980, are summarized below: 
Classroom 



Instrixrtional staff 
Materials 

Projected costs for 1980 

PREST . 

Terminals 

Usage Charge 

Maintenance 

Instructors 

Subtotal 

Cost avoidance 

Projected costs for 1980 



$ 72,9^^0 
$ 2,188 

$ 75,128 



$ 23,520 

$ 76,320 
$ 10,368 
$ 29,176 

$139,38^^ 

$ 7,022 

$132,362 



(5(a$U,588 each) 



(24 (a $4>,900 each; allocated 
over 5 years) 
($263Aterminal/month) 
($36/terminai/month) 
(2 0 $1*,58S each) 
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Figures I and 2 chart the time-cOurse behavior of the projected costs, in 1980 doUars, 
for military and civilian instructors respectively, assuming either a constant cost for 
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Cost comparisons of program delivery based on civilian 
instructors-10% per year decline in hardware costs. 
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usage charge or a 5 percent decline in this charge. With the former assumption, the costs 
of the alternatives, whether based on using military or civilian instructors, do not reach a 
break-even point during the 1980s. With the latter, the costs based on using military 
instructors would break even about 1990; and the costs based on using civUian instructors, 
in 1987. 

9 

DISCUSSION AND CONCLUSIONS 

Determining whether a proposed program is cost-effective hinges on the costs of 
alternative programs. In this study, the classroom approach to ART was found to be more 
cost-effective than PREST because of the relatively low cost of military instructors. 
Higher instructor costs would have led to different conclusions. For example. In a recent 
Army study on the cost-effectiveness of PLATO compared to classroom basic skills 
instruction, it was concluded that PLATO was a cost-effective alternative to traditional 
instruction (Blevins ic Phipps, 1980). However, it must be noted that this study did not 
include costs for the instructor mafiaging the PLATO classroom. These costs, which 
covered a total Of 9«08 instructional hours (.56 Army enlistees and 168 hours per student) 
were included in a $44,000 contract to a public school system to teadi basic skills to 
Army enlistees. This amount of instruction would have cost about $12,000 at ART 
Orlando if military instructors were used. Although the Army study considered the 
PLATO system to be instructor independent, this effort considered that PREST required 
one instructor per 2<f students. Finally, it should be noted that, if the cost-comparison 
had been cc^>ducted in San Diego, where salaries for dvUian instructors are high«' than m 
Orlando, the break-even point for the PREST and classroom approaches with civilian 
instructors would have occurred in about 1984. These examples illustrate some problems 
with conclusions from cost-effective studies (c.f., Kazanowski, 1968). 

The usage charge for PLATO with its centralized mainframe is the main reason why 
PREST costs more than does the classroom approach to ART. Although the mainframe 
will continue to be used in cases where instruction must be managed for a large niimber of 
students or a large mix of courses is offered fdr a limited time, advances in computer 
technology are rapidly making the stand-alone systems more economically attractive than 
the centralized mainframe (MicheU, Morris, & Swope, 1980). In the ART environment, 
where the instruction is fixed throughout the year, it does not seem advantageous to pay a 
subscription fee continually for a limited amount of courseware and to send tiie same 
courseware repeatedly over phone Unes. This problem wiU be partly remedied by thfe 
introduction of a PLATO system that uses microprocessors and floppy discs. This 
configuration stiii requires communication to the mainframe for test and management 
functions, possibly over a reserved terminal. Furthermore, the durability of the floppy 
discs containing the basic skills courseware must be determined before a cost analysis 
could be performed. This new configuration, however, will undoubtedly Improve the coft- 
effectiveness of computer-based instruction. ; " j 

Overall costs could also be decreased if the instructor staff could be decreased 
without a corresponding decrease in program effectiveness. In the PREST approach to 
ART, instructors route students to "bff-line materials and provide feedback regarding 
these materials. These off-line instructor tasks increase instructor dependence that, in 
turn increases the cost of the overall configuration. /Thus, if CAI ^ere employed only in 
those ART modules In which the computer Is instructor Independent, savings due to 
decreased instructor costs could be realized. The extent to which this Instruction could 
be delivered on low-cost microcomputers would lead to further cost benefits, wisher 
(1980 a, b) tested an instructor-independent CAI approach to the phonics and vocabulary 
modules of ART. Results showed that the CAI phonics instruction was as efficient as the 
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classroom instruction, and that the CAI vocabulary instruction was three times more 
efficient than the classroom instruction. Thus, it appears that if, for example, the highly 
efficient vocabulary instruction were delivered on a low-cost device, the system would be 
highly cost-ef fec^e today. This assumes that one instructor is eliminated, and the costs 
of the hardware ate allocated over 5 years. The technology is now available to deliver 
cost-effective training for some of the ART modules* 

This evaluation of PREST did not test a nonnative English speaking population. Thus, 
it is not known how effective PREST would be for them. Since this population is an 
increasing concern in the Navy (Salas, Kincaid, 3c Ashcroft, 1980), plans are being made to 
develop and implement- -in FY 81 --a separate ART module for them. Any computer- 
based system being considered as a substitute for classroom instruction must be suitable 
for this population. 

Because of the varying month-by-month throughput, savings could probably be 
realized by turning off some terminals during the winter months. The increase in ART 
entrants during the summer months would require that instruction be staggered through- 
out the day or that the off-line materials receive more usage. In the classroom, the 
student/instructor ratio fluctuates during the year. The cost for the varying throughput, 
however, affects the classroom and PREST configurations equally. 

The off-line worksheets would be an excellent addition to the ART study skills 
module. Since they are Navy relevant, they would complement the current civilian- 
relevant classroom materials. 

RECOMKAENDATIONS 

1. The PREST program should not be substituted for the current ART classroom 
approach at this time. 

2. Consideration should be given to (a) replacing the vocabulary and comprehension 
modules of ART with a computer-based system and (b) using low-cost, stand-alone 
microcomputers as the delivery devices. 

3. The contractor-developed worksheets should be incorporated into the ART study 
skills module. 

^. The NRRT should be developed further as a selection device for ART. 
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